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Background / Context: The National Center for Research on Early Childhood 
Education’s (NCRECE) program of research is a series of experimental studies of specific 
approaches to training early childhood (EC) educators to be effective in implementation of 
curriculum and instructional interactions focused on promoting language and literacy skills, two 
domains that operate as gatekeepers to later achievement. Previous NCRECE work has 
demonstrated that a 14-week course designed to enhance teachers’ use of effective teaching 
practices was effective in changing teacher beliefs, knowledge, and observed practice (Hamre et 
ah, 2010). This paper examines the extent to which teachers from a variety of backgrounds were 
engaged in the course and the extent to which their engagement was associated with significant 
changes in belief, knowledge, and practice. Below, we briefly discuss the context for this work. 

By age 5, an unacceptably large number of children in- and near-poverty are lacking in 
competencies fundamental to their school success (Duncan et ah, 2007; Eantuzzo et al, 2007; 
Raver, 2008). The long-term effects of early gaps in achievement and social functioning are so 
pronounced that effective and efficient interventions targeting these gaps in the preschool period 
increasingly are viewed as essential to the developmental success of children as well as the 
economic and social health of communities (Heckman, 2006; Heckman & Masterov, 2007; 
Magnusson, Ruhm, & Waldfogel, 2007). 

Several factors have contributed to a growing interest in targeting these interventions 
toward improvements in the quality of teachers’ interactions with children. Eirst, there is now 
compelling empirical evidence that one of the most salient aspects of EC programs’ effects on 
children’s development concerns the nature and quality of teachers’ interactions with children 
(Dickinson & Brady, 2006; Howes et ah, 2008; Jackson et ah, 2006; Mashburn et ah, 2008). 
Research using the Classroom Assessment Scoring System (CEASS: Pianta, EaParo, & Hamre, 
2008) documents the ways in which teachers’ everyday social and instructional interactions are 
linked to growth in early literacy, language, math, and social skills (Curby et ah, 2009; Mashburn 
et ah, 2008). Second, national data suggest that the average pre-k child is likely to experience 
teacher-child interactions of mediocre to low quality (EaParo et ah, 2009; Phillips, Gormley, & 
Eowenstein, 2009; Pianta et ah, 2005). 

A new generation of controlled evaluations, suggests that intensive professional 
development supports that directly target improvements in teacher-child interaction can be 
effective (Bierman et ah, 2008; Domitrovich et al., in press; Pianta et ah, 2008; Ramey & 

Ramey, 2008; Raver et al., 2008). Most of these interventions provide some combination of 
curriculum and classroom-based coaching or mentoring to teachers (Bierman et al., 2008; 
Domitrovich et ah, in press; Pianta et ah, 2008; Raver et al., 2008). Par fewer studies have 
systematically tested the effects of coursework on teacher-child interactions or child outcomes 
(see Dickinson & Caswell, 2007; Howes et al., 1998; Kontos et ah, 1996 for exceptions). 
Coursework has several advantages over coaching or mentoring approaches, particularly related 
to use at scale. Coursework, which can be delivered for cohorts of students, is likely to be 
considerably less expensive than mentoring, which is typically delivered as a 1-on-l 
intervention. Purthermore, effective courses have the potential of being folded into existing 
systems of higher education, in contrast to mentoring approaches which will remain as add-ons 
requiring significant resources from programs. 

Previous work demonstrated the efficacy of a course focused on improving the quality of 
teachers’ interactions with children among 440 early childhood teachers (Hamre et al., 2010). 
Compared to teachers in a control condition, those exposed to the course reported more 
intentional teaching beliefs and demonstrated greater knowledge of and skills in detecting 
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effective interactions. Furthermore, teachers who took the course were observed to demonstrate 
more effective emotional and instructional interactions. The course was equally effective across 
teachers with less than an associate’s degree as well as those with advanced degrees. 

To scale this type of intervention, however, it is important to examine the extent to which 
it can be offered consistently as well as the extent to which a variety of teachers may respond, or 
not, to the intervention. We were particularly interested in the extent to which fidelity of 
implementation and teachers’ participation in and responsiveness to the course content may help 
explain variation in response among those who were assigned to the treatment group. 

Purpose / Objective / Research Question / Focus of Study: First, we examine the 
extent to which there was variation in implementation and engagement across the 15 course 
sections in 9 sites. Second, we examine the extent to which teachers from a variety of 
backgrounds engaged equally in the course, both in terms of quantity and quality of engagement? 
Finally, we examine which aspects of participation are most closely associated with changes in 
teachers’ belief, knowledge, and practice? 

Setting: The NCRECE course was offered in nine sites across the country: Charlotte, 
North Carolina; Chicago, Illinois, Columbus, Ohio; Dayton, Ohio; Hartford, Connecticut; 
Memphis, Tennessee; New York City; Rhode Island; and Stockton, California. Participating 
teachers worked in a variety of EC programs including Head Start, preschool, and child care. 

Population / Participants / Subjects: 137 teachers participated in the course. Teachers 
were diverse in terms of their racial/ethnic and educational backgrounds. Table 1 presents 
descriptive on the teachers in the sample. 

Intervention / Program / Practice: The course, entitled Support of Language and 
Literacy Development in Preschool Classrooms through Effective Teacher-Child Interactions 
and Relationships, was designed to increase teachers’ knowledge about the vital role that 
teacher-child interactions play in learning and skill acquisition and to build specific skills for 
observing teacher-student interactions that contribute to language and literacy skills. Because 
language and early literacy skills are the gatekeepers for later school success, this course also 
provided teachers with the knowledge they need to implement language and literacy curricula 
through effective teacher-child interactions. The course was delivered in 14, 3-hour long 
sessions, through collaborations with local colleges and universities in each site. There were 
between 5 and 15 teachers in each course section. Instructors (n = 15) were provided with 
instructor manuals which included PowerPoint presentations, videos, and written assignments for 
each course section. Instructors attended a week-long training and were provided with ongoing 
implementation support by NCRECE staff. 

Course Instructors were trained to reliability on the CLASS, and on course content and 
implementation, so that consistent delivery was ensured across course sections with a high 
degree of fidelity. Instructors participated in training on the guiding principles, research 
background, and course materials, including the PowerPoint presentations, website, readings, 
grading and homework procedures. Instructors completed a written assignment related to these 
materials for each unit of the course. Weekly individual support calls, and periodic group calls 
were held with instructors, which provided a time to clarify implementation issues, and share 
success and challenges to teaching the course. 

Research Design: The overall study employed a randomized control trial methodology. 
Teachers within each site were randomly assigned to receive the course or to be in a business as 
usual control group. The current study is a non-randomized follow up analysis of 
implementation. 
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Data Collection and Analysis: 

Intervention Implementation Fidelity 

Instructor Adherence: Instruetors taped themselves teaching 5 of the 14 course units and 
a course developer expert coded the implementation on a 3 -point scale (0=not observed to 
2=consistently observed) for 5 aspects of their teaching: overview of presentation, facilitates 
understanding, content knowledge, discussing video, and instructional climate. Items were 
summed to provide a total instructor implementation quality score. 

Participant Intervention Engagement 

Participant Dosage of Intervention: After each course session, instructors entered into the 
study website who of their registered participants were present in class that day as well as their 
completion of homework assignments. 

Participant Satisfaction with Intervention: Participants were surveyed about their 
satisfaction with the course following completion. Participants responded to items such as: 

“What 1 learned in this course was helpful to my development as a teacher.” (alpha = .83) 

Participant Quality of Participation with Intervention: Participants across all sites 
completed the same mid-term and homework. Instructors were trained with how to score the 
exams and homework, and the participant’s scores were entered into a database. Additionally, 
instructors completed 10 items on all participants indicating their perception of participant effort, 
such as “this teacher worked hard to meet the demands of this course.” 

Participant Outcomes: 

The remainder of data were collected through teacher report (pre- and post-course) and 
observations made using videotaped footage of teachers’ classrooms. Teachers reported on their 
beliefs and knowledge regarding effective teacher-child interactions and effective literacy and 
language instruction. Teachers in both conditions sent in four videotapes of their instructional 
interactions over the course of the semester. Measures included: 

Beliefs. The Beliefs about Intentional Teaching scale is a 13-item, 5-point Likert scale 
survey that aims to assess the extent to which teachers believe that children’s learning is 
contingent upon intentional interactions in the classrooms. The Beliefs about Importance of 
Specific Skills-MTP L/L Objective Beliefs asks teachers to report how important literacy and 
language skills (e.g. maintaining 2 turns in a conversation; identifying the first letter in their 
name) are to children entering kindergarten. 

Knowledge. A 15 -item scale which tests a teacher’s understanding of and knowledge about 
the types of interactions with children that lead to positive development {Teachers ’ Knowledge 
of Effective Teacher-Child Interactions-Overall Assessment). A set of multiple choice items 
provide a brief classroom scenario, and then ask teachers to choose the best answer from a set of 
four. Knowledge about language/literacy skills was assessed through 12 items in which teachers 
had to categorize skills into one of six language/literacy domains: alphabet knowledge, print 
concepts, vocabulary and linguistic concepts, pragmatics and social language, narrative skills, 
and phonological awareness. Scores are computed for percent correct within each domains 
Delivery. The quality of teachers’ interactions with children during these videotaped 
activities will be assessed using the CLASS (Pianta, LaParo, & Hamre, 2008). The CLASS 
measures 1 1 dimensions of interactions using 7-point scales: (a) positive climate, (b) negative 
climate, (c) teacher sensitivity, (d) regard for student perspectives, (e) behavior management, (f) 
productivity, (g) concept development, (h) instructional learning formats, (i) quality of feedback, 
(j) language modeling, and (k) literacy focus. Interrater agreement (ratings within 1 point on the 
1 to 7 rating scale) for these dimensions ranged from .75 to .92. 
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Analyses presented below include descriptive analysis of implementation and engagement 
measures as well as hierarchical linear models with teachers nested in course sections. The first 
set of HLM analyses use teacher level demographic factors and beliefs to predict course 
participation. The second set of HLM analyses use teacher- level participation to predict changes 
in teachers’ beliefs and knowledge from pre- to post-course. This second analysis was only 
possible in the first cohort of teachers as pre-test measures were dropped in the second cohort 
due to constraints on teachers’ time. Future analyses will add course section-level predictors to 
each of these models (e.g. number of participants, instructor quality, etc). Finally, CLASS data is 
currently being coded for both cohorts and these data will be analyzed using growth models for 
the final presentation. 

Findings / Results: 

Coding of videos of the course section revealed that instructors delivered the course with 
high levels of fidelity (mean=L99, SD-.02, range 1.95-2). Teacher engagement was also high. As 
displayed in Table 2, on average teachers attended 12.34 of the 14 sessions and completed the 
vast majority of their homework assignments. These attendance and homework variables were 
highly correlated and were composited for further analyses (alpha = .89). Instructors also 
reported high levels of engagement across teachers and teachers overwhelmingly reported that 
the course was useful to their practice. There was more variability in the quality of engagement, 
as measured by scores on homework assignments and the midterm exam. 

There was, however, some evidence of systematic variation in teacher engagement across 
course sections. Interclass correlations (Table 3) suggest that between 0 and 48% of the variance 
in teacher engagement variables was explained at the course section level. There was no course 
section-level variance in teacher-reported usefulness and a very high level of course section-level 
variance in midterms. This latter finding suggests that instructors may have varied quite 
substantially in how they graded midterms. More moderate, but still significant levels of course 
section variance were observed in teachers’ engagement, as observed and as reported by 
instructors, as well as in the quality of their video homework assignments. 

Table 3 presents results of the prediction of engagement from teacher factors. The 
strongest finding was that teachers who reported more authoritarian, teacher-centered, beliefs 
prior to the start of the course were less engaged - they attended fewer session and completed 
fewer assignments, instructors reported them to be less engaged, they reported the course to be 
less useful, and the quality of their work was lower. A second consistent finding was that 
teachers with more experience teaching pre-k were less engaged - they reported to course to be 
less useful and the quality of their engagement was lower. There were not significant differences 
in teacher engagement based on education levels or whether or not they taught in a public school 
building. Head Start teachers did perform significantly more poorly on the midterm than did 
other teachers, but they had equal levels of participation as other teachers, reported the course to 
be as useful, and did equally well as other teachers on homework assignments. Future analyses 
will add course-section level predictors to these models. 

The final analyses examined the extent to which these engagement variables predicted 
changes in teachers’ beliefs and knowledge from pre-course to post-course. Future analyses will 
similarly examine changes in observations of teachers’ interaction with children in the 
classroom. Initial results suggest that teacher attendance and homework completion were an 
important factor in changing beliefs and knowledge. Teachers who had higher level of 
attendance and homework completion displayed greater increases in their knowledge about 
language and literacy skills and effective interactions and also reported increased beliefs 
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regarding the importanee of teaehers taking an active role in children’s learning. There was not 
systematic evidence for associations between any of the other engagement variables and changes 
in teacher belief and knowledge. 

Conclusions: Despite a clear need for improvement in the quality of EC teachers 
interactions with children, programs and teachers remain uncertain about how to make 
systematic changes in the nature and quality of these classroom interactions. The professional 
development system built to address this problem has a history of incoherence and 
ineffectiveness (Ball & Cohen, 1999). Recent work suggests that coursework focused 
specifically on improving teachers’ knowledge and skills regarding interacting with young 
children can be effective (Hamre et ah, 2010). The findings reported here provide further 
evidence to support the effectiveness of this course, but also provide some interesting insights 
into groups of teachers for whom this course may be of greatest interest. 

The course was implemented with high levels of fidelity across 9 sites and 15 different 
instructors. Observations of the course sections revealed that most instructors were using 
materials as intended, completing all components of the course, and actively facilitating 
conversations with teachers. This provides evidence that coursework can be delivered 
consistently at scale. There were, however, variations in teachers’ engagement across course 
sections. Future analyses will examine the extent to which course section-level factors, such as 
class size or the composition of students, may contribute to this finding. 

Results suggest several teacher-level factors that predict engagement. Teachers with less 
authoritarian beliefs and teachers with less pre-k teaching experience were more likely to be 
engaged in course material. It may be that the course content, which focused on the importance 
of teacher-child interactions and also highlighted the importance of fostering children’s 
autonomy, was in conflict with some teachers beliefs in ways that led them to become more 
disengaged with course material. Teachers with more teaching experience reported the course 
was somewhat less useful, although they still gave generally high ratings. Interestingly, the 
quality of experienced teachers’ engagement was somewhat lower - they performed more poorly 
on homework and midterm assignments. This is somewhat counterintuitive as we might expect 
these teachers to know the most about classroom interactions. However, it may be that teachers 
who have been in the classroom for a long time have more set ideas about what effective 
interactions look like, and thus have a harder time performing on tasks that ask them to shift to 
thinking about new ways to interact in the classroom. 

Teachers from a wide variety of educational backgrounds reported that the course was 
useful and displayed equal level of engagement. It is unusual for a course to target such a broad 
range of teachers but this finding suggests that the course material, focused on teacher-child 
interactions, was relevant to teachers across the educational spectrum. This may reflect the fact 
that few teachers, even those who have taken many early childhood courses, have had 
opportunities to learn about effective teacher-child interactions in a college course setting. 

Adding to efficacy evidence from the intent-to-treat analysis, this study suggests that 
teachers who attended more course sections and completed more of the homework assignments 
displayed greater changes from pre- to post-test on three of the four measures of beliefs and 
knowledge. Thus, dosage appears to matter. There was not evidence to support the idea that 
teachers’ own views of usefulness, instructor-reported engagement, or the quality of teachers 
engagement (i.e. homework and midterm scores) were systematically associated with changes in 
belief and knowledge. Future analyses may examine the individual components of participation 
and quality of engagement in more detail to try to unpack the “active ingredients” of this course. 
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Appendix B, Tables and Figures 

Table 1. Teacher Demographics (n=137) 



% 


Female 


97% 


Ethnicity 




African American 


44% 


Asian 


2% 


Caucasian 


10% 


Latino 


33% 


Other 


10% 


Highest Education 




AA or less 


42% 


BA 


44% 


More than BA 


14% 


Head Start Teacher 


59% 


In Public School Building 


31% 



Table 2, Descriptive Statistics on Implementation Variables (n=137) 





Mean 


SD 


Range 


Number of Course Sessions Attended 
Number of Homework Assignments Completed 


12.34 


1.98 


2-14 


Video 


16.44 


3.65 


0-18 


Guided Reading 


5.43 


1.4 


0-6 


Papers 


1.80 


.50 


0-2 


Instructor Reported Engagement 


4.22 


.83 


0-5 


Teacher Reported Usefulness 


4.63 


.51 


1.8-5.0 


Quality of Video Homework (% correct) 


82.94 


14.29 


33-100 


Midterm Exam Scores 


47.86 


8.31 


23-60 
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Table 3. Prediction of Engagement Variables from Teacher Background and Beliefs (N = 137) 







Instmctor 


Teacher- 


Quality of 






Observed 


Reported 


reported 


Video 






Participation 


Engagement 


Usefulness 


Homework 


Midterm 


ICC 


.12 


.08 


.00 


.16 


.48 




B(SE) 


B(SE) 


B(SE) 


B(SE) 


B(SE) 


Intercept 


.39 (.37) 


5.43 (.29) 


5.08 (.17) 


104.92 (5.56) 


62.60 (3.0) 


AA or less 


-.04 (.28) 


-.06 (.21) 


.12 (.12) 


-.71 (4.06) 


-2.14 (2.0) 


BA 


.10 (.24) 


.08 (.19) 


.07 (.12) 


1.98 (3.55) 


-0.15 (1.7) 


Pre-K 

Experience 


.02 (.01) 


.01 (.01) 


-.01 (.005)* 


-.32 (.16)* 


-0.16 (.08)’ 


Head Start 
Teacher 


-.16 (.18) 


-.04 (.14) 


.06 (.08) 


2.59 (2.73) 


-2.93 (1.37)* 


In Public 
School 


.30 (.19) 


.08 (.14) 


-.14 (.08) 


4.59(2.83 


0.89 (1.47) 


Authoritarian 

Beliefs 


-.26 (.13)’ 


-.49 (.10)’*’ 


-.15 (.06)’ 


-8.64 (1.95)’*’ 


-4.39(1.47)**’ 



* ^4; *4= * 

p<-05; p<-01; p<.001 



Table 4. Prediction of Gains in Teacher Knowledge and Beliefs from Engagement Variables 
(n=69) 





Beliefs about 
Importance of 


Knowledge of 








Literacy & 


Literacy & 


Beliefs about 


Knowledge of 




Language 


Language 


Intentional 


Effective 




Skills 


Skills 


Teaching 


Interactions 


ICC 


.00 


.11 


.00 


.12 




B(SE) 


B(SE) 


B(SE) 


B(SE) 


Intercept 


2.56 (.54) 


53.56 


.84 (.75) 


49.89 


Pre-Course 

Score 


.30(.09)” 


.09 (.09) 


.54 (.10)*” 


.14 (.14) 


Observed 

Participation 


.00 (.10) 


8.09 (3.64)* 


.34 (.12) ** 


8.15 (3.75)’ 


Instmctor 

Reported 

Engagement 


-.06 (.10) 


-2.63 (3.17) 


.00 (.13) 


1.89 (3.61) 


Teacher 








-4.51 (3.55) 


Reported 

Usefulness 


-.01 (.09) 


-.49 (3.51) 


.01 (.12) 




Quality of 
Video 
Homework 


.00 (.00) 


.09 (.16) 


.01 (.01) 


0.05 (.16) 


Midterm 


.02 (.01)* 


.45 (.28) 


.00 (.01) 


0.52 (.28) 



^ ^ :k ^ ^ H: 

p<-05; p<-01; p<.001 
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